#database architecture

[bogkeep]

in the most recent patreon exclusive bonus episode of the very good podcast "if books could kill" michael hobbes has a very long tangent about a separate book he read for research (manhood in the making by david d gilmore) which is like, an anthropological study in what manhood means in various cultures across the world, and I Need This Book So Bad. i crave anthropological analysis of socially constructed gender roles in non-western cultures. but i must be patient. i found this book on a swedish bookstore but at the cost of too much money for a single book, and for a much more agreeable price on a norwegian bookstore - but they only ship to norway. in two months i will return. surely i can wait to months. i shall be so good and so patient

[thetrueparanormal]

A new paranormal artifact article documenting the Crying Boy paintings has been uploaded to the world's largest database of paranormal research 😄

[kicksnscribs]

driving myself fucking insane bc im trying to find reference to model a thing and for the life of me I cannot find this specific roof shape bc i KNOW i saw it somewhere but now im doubting myself bc nothing i type in give me any results whyyy

[1o1percentmilk]

i don't even want to take half my classes that im registered for autumn quarter

[yj-98]

hm.

[goodoldbandit]

Data Unbound: Embracing NoSQL & NewSQL for the Real-Time Era.

Sanjay Kumar Mohindroo Sanjay Kumar Mohindroo. skm.stayingalive.in Explore how NoSQL and NewSQL databases revolutionize data management by handling unstructured data, supporting distributed architectures, and enabling real-time analytics. In today’s digital-first landscape, businesses and institutions are under mounting pressure to process massive volumes of data with greater speed,…

[excelworld]

[projectchampionz]

Explore These Exciting DSU Micro Project Ideas

Explore These Exciting DSU Micro Project Ideas Are you a student looking for an interesting micro project to work on? Developing small, self-contained projects is a great way to build your skills and showcase your abilities. At the Distributed Systems University (DSU), we offer a wide range of micro project topics that cover a variety of domains. In this blog post, we’ll explore some exciting DSU…

[nitor-infotech]

Understanding Database Sharding

Imagine you have a library with thousands of books, and finding a specific one becomes time-consuming. To speed things up, you decide to split the collection into smaller sections based on genres, making it quicker to locate any book.  

Similarly, Database Sharding divides a large database into smaller, more manageable pieces to improve performance and scalability. 

Fig: Database Sharding Banner Image 

What is a Sharding Database 

Database sharding is like storing a huge database across several machines. Imagine one server trying to handle all the data—it can only do so much before it starts slowing down. By splitting the data into smaller chunks, or shards, and storing these across multiple servers, we can manage and process large amounts of data more efficiently. 

As an application grows, more users and data can turn the database into a bottleneck, slowing everything down and frustrating users. Sharding also helps by allowing parallel processing of these smaller datasets, keeping things running smoothly even as demand increases. 

Scaling Techniques in Database Sharding 

Scaling database sharding involves several techniques to ensure efficient management and distribution of data. Here are some key methods: 

Horizontal Partitioning 

This technique involves splitting the data across multiple servers based on a sharding key, such as user ID or geographic region. Each server, or shard, contains a subset of the overall data. This approach allows the system to scale out by adding more servers, thereby distributing the load and improving performance and reliability. 

Vertical Partitioning 

This technique divides the database into smaller tables, each stored on different servers. Each server handles a specific aspect of the application, such as user profiles, transactions, or product details. By separating the data based on functionality, vertical partitioning can improve query performance and make it easier to manage and scale specific parts of the application independently. 

Range-Based Sharding 

Distributes data management based on a continuous range of values. For example, user IDs 1-1000 on one shard, and 1001-2000 on another. 

Hash-Based Sharding 

Uses a hash function on the sharding key to evenly distribute data across shards. This helps avoid uneven data distribution. 

Directory-Based Sharding 

Maintains a lookup table or directory that maps each data item to its corresponding shard. This allows flexible and dynamic distribution of data. 

Each technique has its advantages and is chosen based on the specific needs and growth patterns of the application. 

Benefits of Database Sharding 

Database sharding offers several benefits: 

Improved Performance: By distributing the data across multiple servers, each server handles a smaller subset of the data, reducing the load and improving query response times. 

Scalability: Sharding allows horizontal scaling, meaning you can add more servers to handle the increased load, making it easier to scale the database as the application grows. 

Increased Availability: With data distributed across multiple servers, the system can continue to operate even if one shard fails. This redundancy enhances the overall availability and reliability of the application. 

Efficient Resource Utilization: Different shards can be optimized for specific workloads, allowing better use of hardware resources. For instance, high-traffic shards can be allocated more resources, while less busy shards use fewer resources. 

Reduced Maintenance: Smaller databases are easier to back up, restore, and maintain. Sharding breaks down the database into more manageable pieces, simplifying administrative tasks. 

Factors to consider before Sharding 

Before deciding to shard your database, consider the following factors: 

Database Size: Sharding is typically suitable for large databases that have outgrown the capacity of a single server. 

Traffic Patterns: If your database experiences uneven traffic patterns, sharding can help balance the load. 

Growth Projections: If significant future scaling is anticipated, sharding can be a beneficial strategy. 

Complexity: Sharding introduces additional complexity to your database architecture and requires careful planning and ongoing maintenance. 

Cost: Sharding can be costly due to the need for extra hardware resources and infrastructure to support multiple servers. 

So, database sharding offers both advantages and challenges, and it is important to determine if it aligns with your application’s requirements. 

To know more about database management, reach out to us at Nitor Infotech. 

[techdirectarchive]

How to create an Amazon Relational Database Service Instance

Amazon Relational Database Service is a fully managed database service in the cloud. It offers capabilities for setting up, operating, and scaling relational databases with ease. In this article, we shall discuss How to create an Amazon Relational Database Service Instance. Please see How to install Amazon RDS using Terraform, how to Migrate Veeam MSSQL Database to a new MSSQL Server, and How to…

[thetrueparanormal]

A new article documenting Ammersoyen Castle, located in the Gelderland province of the Netherlands, has been uploaded to the world's largest database of paranormal research 😄

http://thetrueparanormal.blogspot.com/2025/05/ammersoyen-castle.html

[ezrasf]

I am putting this senior data systems architect position out into the aether for selfish reasons.

I'm not the hiring manager, but I'm hoping this gets filled so I can stop worrying about the work getting shifted onto my unqualified plate. #dataarchitect

[mamtarajshree]

Client Server Database Architecture

Client Server Database Architecture Client server database architecture consists of two logical components. One is “Client” and the other one is “Server”. Clients are those who send the request to perform a specific task to the server. Servers are normally receive the command sent by the clients, perform the task and send the appropriate result back to the client. Example of client is PC where as…

[ravinskiy]

Brilliant technique for handling database queries literally saved Discord

It helped them store trillions of messages and fetch them without bringing their DB cluster to its knees.

The technique is called Request Coalescing.

And it’s too good to ignore.

But what’s so special about it?

If multiple users are requesting the same row at the same time, why not query the database only once?

This is exactly what Request Coalescing helps us achieve.

Here’s what happens under the hood:

- The first user that makes a request causes a worker task to spin up in the data service

- Subsequent requests for the same data will check for the existence of that task and subscribe to it

- Once the initial worker task queries the database and gets the result, it will return the row to all subscribers at the same time.

There are several pros to using Request Coalescing:

- Efficient utilization of database resources

- Ability to handle more concurrent requests without creating hot partitions

- Reduce latency

But there are some cons as well:

- Implementation can be complex with regards to getting a fair distributed reader-write lock. Basically, multiple readers need to access the data simultaneously while preventing conflicting writes

- Overall latency may go down, but certain requests will take more time

Of course, this technique is NOT needed normally. But at a certain scale, it can actually save your business.

Credits to Saurabh Dashora

[jannah-software]

DevOps with Artificial Intelligence, Automation, and Blockchain: Introductory Part 7

Video Highlights: This marks the conclusion of our introductory videos. Watching Jannah’s services come into ‘running’ state with the following command: kubectl get -w pods -A Counting the number of services that are up and running with: kubectl get -w pods -A | wc -l Take home points: We have Ansible Playbooks. The Playbook roles contain tasks to deploy our infrastructure to a K8…